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ABSTRACT: One critical aspect that hinders the systematic use of concept 
mapping in everyday classrooms is the difficulty of providing high-quality 
feedback to students so as to keep improving and revising their concept maps 
(Cmaps). The development of an innovative way to analyse, at a glance, students’ 
Cmaps is presented to allow a diagnostic assessment of student conceptual 
learning. This paper proposes Neighbourhood Analysis (NeAn) as an innovative 
way to foster meaningful learning through the identification of Limited or 
Inappropriate Propositional Hierarchies (LIPHs) in Cmaps. The instructional 
strategy that underlies NeAn involves the selection of a compulsory concept 
(CC), which must be a threshold concept addressing the Cmap’s focal question. 
The goal is to estimate students’ understanding of the subject, in order to detect 
the presence (or absence) of LIPHs. We analysed 69 Cmaps from a higher 
education setting and found that 175 propositions were related to the CC. This 
subset of all propositions was enough for instructors to provide specific high- 
quality feedback to their students, even under normal teaching conditions. NeAn 
is a straightforward way to identify LIPHs allowing instructors to make 
diagnostic assessments of students’ conceptual outcomes during the learning 
process, and to track the effects of their instructional options. NeAn can be 
meaningfully applied for assessing science students at secondary level and above, 
with special potential applicability across science subjects. 

KEY WORDS: climate change, concept mapping, diagnostic assessment, 
meaningful learning, misconceptions 


Introduction 

Concept mapping is a well-known technique used to graphically represent 
aspects of individuals’ mental models (Novak, 2010). Concept maps 
(Cmaps) are formed by embedding a set of concepts into a propositional 
network. Each proposition is formed using two concepts, which are linked 
by an arrow to indicate the reading direction (initial concept - linking 
phrase —> final concept). A clear explanation of the relationship between 
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these concepts must be added above or below the arrow to let the reader 
identify the precise understanding of the concepts held by the map-maker. 
The inclusion of linking phrases to clarify conceptual relationships makes 
Cmaps more powerful than other graphical techniques used to represent 
knowledge and information (Davies, 2011). Additionally, because our 
short-term memory can process textual and graphical information 
simultaneously (Paivio, 1990; Sweller, Ayres & Kalyuga, 2011), Cmaps 
are accessible even to first-time users. The process of making learning 
visible using Cmaps is therefore valuable in guiding the instructional 
process and assessing student conceptual outcomes (Kinchin, Hay & 
Adams, 2000; Hay, 2007; Hay, Kinchin & Lygo-Baker, 2008; Hay, Wells 
& Kinchin, 2008; Ingeq, 2009; Jaber & BouJaoude, 2012). Cmaps also 
makes visible the pedagogic resonance, i.e. the degree of mismatch 
between teacher’s expert knowledge and students’ learning. Therefore, 
students’ Cmaps are suitable for collaborative discussions and reflections 
within the specific context of a course, providing an interesting 
documentation of the teaching-learning process (Kinchin, Lygo-Baker & 
Hay, 2008). 

Since 1990, when the Journal of Research in Science Teaching 
published a special issue on concept mapping (see, Novak, 1990; Wallace 
& Mintzes, 1990;Wandersee, 1990), several reports in the literature have 
confirmed the usefulness of concept mapping in science education 
(Markham, Mintzes & Jones, 1994; Markow & Lonning, 1998; Rye & 
Rubba, 1998; Laight, 2004; Van Zele, Lenaerts & Wieme, 2004; Vassilis, 
Marida & Vassiliki, 2007; Oliver, 2009; Conradty & Bogner, 2010; 
Gerstner & Bogner, 2010; Schaal, Bogner & Raimund, 2010; Correia, 
2012). Assessment and the development of score systems for Cmaps have 
deserved special attention due to the critical role of the feedback provided 
by the instructor to keep students continuously revising and improving 
their own Cmaps (Ruiz-Primo & Shavelson, 1996; Rice, Ryan & Samson, 
1998; McClure, Sonak & Suen, 1999; Ruiz-Primo et al., 2001). However, 
a fast procedure to allow the diagnostic assessment of students’ 
conceptual knowledge from their Cmaps is still necessary to take full 
advantage of concept mapping, mainly under normal teaching conditions. 
We believe there is a methodological gap that has not been addressed 
adequately to develop new strategies to use Cmaps in classrooms. As 
pointed out by Mayer (2010), there is an emergent science of instruction 
that can help us to devise innovative teaching strategies from a research- 
based theory. This paper is an attempt to fill this gap and to leverage the 
current experiences with Cmaps for students and instructors from an 
informed perspective about concept mapping, instruction and learning 
processes. In our opinion, this methodological gap complements the 
teaching ecology discussed by Kinchin (2001), which can be understood 
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as a set of conditions to favour the effective use of Cmaps in the 
classroom. 

New concepts are integrated into our cognitive structures to greater or 
lesser degrees depending on the cognitive effort that we expend in seeking 
meaningful relationships between concepts. Methods for relating new 
concepts and propositions to our existing cognitive structures fall along a 
continuum between the two extremes of meaningful learning and rote 
learning (Ausubel, 2000; Mayer, 2002). The former only occurs when 
conceptual relationships are established in a non-arbitrary and non-literal 
way, which requires more effort in order to relate prior knowledge to new 
information. On the other hand, rote learning occurs when these 
relationships are established in an arbitrary and literal way, skipping the 
intentional use of prior knowledge to make sense of the new information 
(Ausubel, 2000; Novak, 2010). There are 3 critical conditions to support 
meaningful learning: 

• the identification of students’ prior knowledge, 

• the selection of instructional materials that are potentially 
meaningful to the students, and 

• the students’ option to learn meaningfully. 

Meaningful learning depends on the idiosyncrasies of each person 
(■ e.g ., prior knowledge, experiences, self-efficacy, values, and control 
beliefs), the instructional strategies used, and learners’ intrinsic desire to 
make meaning (Pintrich, Marx & Boyle, 1993; Ausubel, 2000; Novak, 
2010). Moreover, meaningful learning does not imply the absence of 
conceptual mistakes. Rather, the literature refers to examples of such 
mistakes using misconceptions, alternative conceptions, naive notions, 
and pre-scientific notions. Novak (2002) has proposed the term Limited or 
Inappropriate Propositional Structures (LIPHs) to refer to these kinds of 
conceptual errors. The identification of LIPHs in Cmaps is 
straightforward, as the lack of semantic clarity of some propositions 
indicates the presence of mistakes. Frequently, poorly chosen linking 
phrases limit the accuracy of messages embedded in the propositional 
network. Novak (2002) suggests LIPHs as suitable starting point for 
professors to foster meaningful learning and intentionally plan further 
instructional activities. 

LIPHs can be the result of meaningful learning and changing them is 
therefore a difficult task, in which students gradually revise the relevant 
structures of their own knowledge and build up new propositions over 
time. For instance, they must create new meanings from the comments 
made by the instructor in order to revise the Cmap under evaluation. If 
students choose to use rote rather than meaningful learning to overcome 
their LIPHs, the knowledge involved will not be easily applied in different 
contexts. This knowledge usefulness will be limited in time and restricted 
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to the context used during the learning process (Novak, 2002). One 
educational challenge posed by this approach is convincing students to 
choose meaningful rather than rote learning. High-quality instructor 
feedback during the learning process is critical in order to keep students 
committed to learning meaningfully throughout the course and to develop 
their self-regulative capacities (White & Frederiksen, 1998). For all these 
reasons, we believe that there is a need in the literature for a procedure 
that allows for rapid identification of LIPHs using student Cmaps. 

Neighbourhood Analysis 

Neighbourhood Analysis (NeAn) is an innovative way to identify LIPHs 
in Cmaps. The instructional strategy that underlies NeAn involves the 
selection of a compulsory concept (CC) to be used by the students during 
the construction of their Cmaps. The CC is selected by the instructor, as is 
the focal question to be addressed by the map-makers. Some criteria to 
make an appropriate choice of CC are: 

• the selection of a threshold concept, which may be 
transformative, integrative, irreversible, troublesome and/or that 
result in a troublesome knowledge as proposed by Meyer & 
Land (2005), 

• the in-depth discussion of the concept during the didactic 
activities, and 

• the usefulness of the concept to address the focal question 
appropriately. 

The CC also becomes a privileged stalling point for evaluating the 
Cmaps’ propositional networks, because all other concepts can be 
classified into neighbour concepts (NC) or supplementary (SC) concepts. 
The former forms a proposition with the CC whereas the latter is not 
directly linked to CC (Figure 1). Therefore, NeAn allows us to classify 
Cmaps’ concepts before reading the propositional network. Instructors can 
use NeAn to make a diagnostic assessment of student learning at a glance, 
considering only the semantic content of the propositions involving the 
CC. 

Concept mapping is usually more cognitively demanding for students 
than most traditional exams, which are based on recalling information as 
presented during instruction. This traditional, prevalent approach 
considers the students to be passive receivers of content presented by 
professors, capable of recording the information without mistakes. Rote 
learning is the most preferred option in this situation because the learning 
expectations involve only the memorization of facts (Kember, 1996; 
Kinchin, Lygo-Baker & Hay, 2008). Alternatively, the construction of 
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Cmaps considers students to be meaning makers who actively transform 
received information according to their prior knowledge (Novak, 2002; 
Novak, 2010). 
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Figure 1. Classification of the Cmaps concepts according to their 
relationship with the CC (black box): NCs (shadowed white 
boxes) are directly linked to the CC, whereas SCs (white boxes) 
do not form a proposition with the CC. 

We believe that inclusion of threshold concepts may make the 
creation of Cmaps even more difficult, because these concepts are not 
chosen by the map-makers themselves. As a consequence, map-makers 
need to find appropriate neighbour concepts (NCs) and linking phrases to 
express how the threshold concept can be inserted into the propositional 
network. This task may be more difficult than establishing propositions 
only using concepts selected by the Cmap’s author. It is possible that the 
threshold concept may not be familiar to some students, in which case the 
appearance of LIPHs is more probable. On the other hand, when the map- 
makers can choose all concepts themselves (i.e., in the absence of a CC), 
the appearance of LIPHs is less probable because the map-makers feel 
more comfortable and use only concepts with which they are familiar, 
thereby avoiding the risk of exposing their own conceptual gaps. In this 
case, it is more difficult for instructors to identify LIPHs and to plan 
further activities to foster meaningful learning. In other words, 
propositions that involve the threshold concept can help to externalise 
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naive messages or conceptual mistakes. In both cases, the identification of 
LIPHs is straightforward during the Cmap evaluation, and it can be done 
at a glance. Therefore, the CC facilitates the appearance of LIPHs, as well 
as diagnostic assessment, which instructors can carry out by comparing 
Cmaps obtained under the same instructional conditions. Figure 2 
summarises the key ideas involved in the use of Neighbourhood Analysis 
(NeAn) as an innovative way to follow up on the learning process, using 
LIPHs as an important piece of information for students and instructors. 
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Figure 2. Concept map to address the following focal question: How can 
NeAn foster meaningful learning by using concept maps? 
Shadowed boxes indicate the potential instructional use of NeAn, 
whereas non-shadowed boxes indicate the strategy that underlies 
NeAn. 
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Aims of this Study 

This paper proposes NeAn of Cmaps as an innovative way to identify 
LIPHs, in order to foster meaningful learning in science education. 
Empirical results are presented to show the usefulness of NeAn in making 
diagnostic assessments of student conceptual outcomes in a higher 
education setting. 


Research Methods 


Data Collection 

The empirical data included Cmaps (n=69) collected during the Natural 
Science Course, which is offered for all first-year undergraduate students 
at School of Arts, Sciences and Humanities, University of Sao Paulo 
(SASH/USP). The main goal of this course is to provide a broad overview 
of the impact of scientific and technological development on our society 
(Correia et al., 2010; Laherto, 2010). 

The data were collected during 2010 and all Cmaps were made 
individually during the second exam for the Natural Science Course (E2), 
which occurred during the tenth class session (Figure 3). A training period 
to increase the students’ proficiency as map-makers was offered during 
classes 1-4, following procedures described in the literature (Aguiar, 
Cicuto & Correia, 2014). 
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Figure 3. Overview of the Natural Science Course, highlighting the 
activities related to the training period on concept mapping 
(classes 1-4) and to the data collection (classes 6-10). 

Three instructional strategies were used during the training activities: 
half-structured Cmaps (HSCmaps), expanded collaborative learning 
(ECL), and propositional clarity tables (PCTs). The HSCmap was inspired 
by the cyclical Cmap and experiments on dynamic thinking described in 
the literature (Safayeni, Derbentseva & Canas, 2005; Derbentseva, 
Safayeni & Canas, 2007). The HSCmap required summarising capabilities 
because it restricted the number of concepts used during Cmap 
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construction. However, since the HSCmap did not define the map’s 
structure, the map-maker was free to build propositions without 
restrictions. 

Expanded collaborative learning (ECL) is characterised by student 
peer review of material that they produce collaboratively (Author, omitted 
reference). Since students inhabit a relatively consistent zone of proximal 
development, peer review offers an opportunity for them to share 
knowledge with each other; this experience is distinct from their 
interactions with the instructor, who is not in the same zone of proximal 
development (Novak, 2010; Vygotsky, 1978). Peer review enhances 
collaborative activities developed for small groups of students, and for 
this reason, we expect ECL to distinctly shift the nature of the learning 
experience and outcomes. The use of ECL can increase student awareness 
of personal achievements and failures in the Cmap training session. 
Moreover, ECL is an assessment exercise that may offer a safe avenue to 
self-evaluation, which allows map-makers to continuously revise their 
own Cmaps. 

The propositional clarity table (PCT) was designed to reinforce the 
Cmap structure and is based on semantic units (Aguiar, Cicuto & Correia, 
2014). The PCT asks the map-maker to go beyond reading and checking 
the Cmap as a whole; rather, he/she is asked to pay close attention to each 
proposition in the map. A four-column table is prepared, wherein each 
row contains one proposition from the Cmap. The first three columns ask 
students to describe the elementary components of the propositions (initial 
concept, linking phrase and final concept), while the last column asks 
students to rank the clarity of each proposition (Is it clear? Yes/No). 

Climate change was the subject discussed during classes 6-10 (Figure 
3). Didactic activities highlighted the role of dispersion as a key concept 
in understanding atmospheric dynamics and spatial aspects related to 
climate change. The literature suggests that there is a lack of 
understanding of how local actions are related to global environmental 
problems (Fenger, 2009; Ungar, 2000). Assigned readings (Mendes, 
2003), news from online mass media, and satellite images were selected 
as instructional materials to be used in the classroom (Table 1). 

Two videos from animated satellite images were showed to the 
students during class 9. The first one presented the Geostationary 
Operational Environmental Satellite and was produced by the National 
Oceanic and Atmospheric Administration. It showed the satellite 
capabilities in tracking oceanic and atmospheric dynamics, measuring 
temperature, humidity and other physico-chemical parameters. The most 
interesting part was the images showing the pollution (dust) emitted in 
California spreading throughout the Pacific Ocean. The second video was 
produced by the National Weather Service (Met Office, UK) to 
demonstrate how fast the ash emitted by the Eyjafjallajokull volcano in 
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Iceland reached the north of Europe. The intensity of the emission was 
sufficient to close many important European airports for several days 
during April, 2010. This event received detailed coverage by mass media 
and occurred during the Natural Science Course. Despite being an 
unexpected video, it was meaningful to the students because it was 
relevant to the broader issue under discussion, through its confirmation 
that the atmosphere is not static: particles and gases can move from one 
place to another, conferring a dynamic nature to the processes that occur 
above the Earth’s surface. The use of both videos allowed students to 
visualise atmospheric dynamics while the professor talked about the 
images briefly. This approach is based on the cognitive theory of 
multimedia learning (Mayer, 2005). 


Table 1. Learning activities on climate change developed during the 
Natural Science Course (NCS) during classes 6-10. 


Class 

Instructional Material 

Classroom Activities 

6 

Text “Environment: 

Science and Technology” 
(Mendes, 2003) 

Lecture classes on and discuss about the 

text 

Analysis of the relationships between 
science, technology and global 
environmental problems 

7 

Text “Environment: 
Energy” (Mendes, 2003) 

Lecture classes on and discuss about the 

text 

Analysis of current standard of energy 
consumption and increased demand for 
fossil fuels 

Integration of the concepts of classes 6 
and 7 through the discussion of climate 
change 

8 

News available on the 
Internet about climate 
change, selected by 
students 

Activity in small groups of students 
Identification of sections of the news that 
indicate relationships between science / 
technology, science / society and 
technology / society 

Discussion about how climate change is 
presented in the media 

9 

Satellite images that 
represent the atmospheric 
dynamics 

View videos that show the dynamic nature 
of atmospheric processes. 

Discussion about the relationship between 
local actions and global effects due to the 
dispersion of gaseous pollutants in the 
atmosphere. 

10 

Cmap prepared for 
consultation during the 
exam (E2) 

Preparation of individual Cmap, with the 
compulsory concept (dispersion) 
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The second exam (E2) involved the construction of a nine-concept 
half-structured concept map (HSCmap) to address the following focal 
question: “How are scientific and technological developments related to 
climate change?” (Figure 4). The students were required to use 
“dispersion” as a compulsory concept (CC) to set up the propositional 
networks of their Cmaps. 


I 


i 

I 






Figure 4. Nine-concept HSCmap used during the second exam (E2) of the 
Natural Science Course. The dashed box highlights the root 
concept of the HSCmap and student knows that this is the 
starting point for readers. 

Data Analysis 

Categorisation of propositions containing the CC 

The 69 Cmaps produced by the students contained 985 propositions. 
However, Neighbourhood Analysis (NeAn) uses the compulsory concept 
(CC) as a privileged starting point to select propositions for analysis. 
Therefore, only the propositions containing “dispersion” (n=175, or 18% 
of the total) were used for the data analysis. 

Four categories of analysis were utilised to classify the propositions 
according to their messages: limited (L), inappropriate (I), relevant (R) 
and very relevant (VR). The category limited (L) was assigned to 
propositions that could not be understood because of the lack of semantic 
clarity. The map-maker was not able to make her/himself clear, mainly 
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due to poor choice of the linking phrase (e.g., dispersion - of —> air 
masses). The inappropriate category (I) was assigned to propositions that 
expressed a mistake involving dispersion (CC), according to current 
scientific knowledge (e.g., dispersion - causes —> warming of the Earth). 

The category R was used to classify propositions showing a relevant 
connection between the NC(s) and the CC. However, these propositions 
did not mention the relationship between dispersion and the global aspects 
of climate change (e.g., dispersion - is a concept coming from —> science; 
atmospheric phenomena - require understanding the concept of —> 
dispersion). Lastly, the category VR was chosen only for propositions that 
mentioned dispersion to refer to some spatial aspect related to atmospheric 
dynamics. VR propositions established clear and valid statements about 
the global aspects of climate change using the CC (e.g., local action - 
requires understanding the concept of —> dispersion; global effects - 
require understanding the concept of —> dispersion). 

Score system for the overall Cmap analysis 

On the basis of the categories assigned for each proposition, the Cmaps 
were analysed using the score system presented in Table 2. The very 
relevant (VR) propositions received the highest score (+2), whereas the 
inappropriate (I) propositions received the lowest (-2). Propositions 
classified as relevant (R) and limited (L) were scored with +1 and -1, 
respectively. 


Table 2. Score system for propositions containing dispersion (CC) based 
on four categories. 


Category 

Score 

Example 

Very relevant (VR) 

+2 

global effects - requires understanding the 
concept of —> dispersion 

Relevant (R) 

+ 1 

dispersion - is a concept coming from —> science 

Limited (L) 

-1 

dispersion - of —> air masses 

Inappropriate (I) 

-2 

dispersion - causes —> warming of the earth 


Each Cmap was scored by considering its weighted averages, 
according to the equation (1), 

weighted averages = [(N V r x 2) + (N R x 1) - (N L x 1) - (N : x 2)] / (N V r + 
Nr + N l + N,) (1) 

where N V r, N r , N l and N, are the number of propositions categorised as 
very relevant (VR), relevant (R), limited (L) and inappropriate (I), 
respectively. The weighted averages values for all Cmaps (n=69) were 
used to compare the Cmaps in order to find patterns and identify LIPHs. 
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Results and Discussion 

Ne An first step: categorisation of the propositions containing the CC 

The subset of propositions (n=175) involving dispersion (CC) was 
carefully considered during the NeAn. Semantic clarity and understanding 
were the parameters taken into account to categorise each proposition. 
Inappropriate (I) and limited (L) propositions were easily detected during 
the categorisation. As a result, limited and inappropriate propositional 
hierarchies (LIPHs) were more evident to the instructor. Table 3 presents 
the frequency of each category, according to the number of propositions 
between CC and its neighbour concepts (NCs). 


Table 3. Categorization of propositions (n=175) containing dispersion 
(CC) as initial or final concept. 


Category 

Score 

Number of Propositions 







1 

2 

3 

4 

5 

6 

9 

Total 

Number of 

- 

7 

33 

17 

4 

4 

1 

1 

67 a 

Cmaps 

Very 

+2 

2 

25 

13 

6 

6 



52 

relevant 









(30%) 

(VR) 

Relevant 

+ 1 

3 

23 

25 

8 

7 



66 

(R) 









(38%) 

Limited 

-1 

- 

7 

2 

- 

4 

3 

7 

23 

(L) 









(13%) 

Inapprop. 

-2 

2 

11 

11 

2 

3 

3 

2 

34 

(I) 









(19%) 

Total 

- 

7 

66 

51 

16 

20 

6 

9 

175 



4% 

38% 

29% 

9% 

11% 

3% 

5% 

100% 


a In spite of declaring the use of dispersion was compulsory, two Crnaps did not presented 
it. Therefore, the presented results were calculated from 67 instead of 69 Cmaps. 


The use of the CC challenged the students to address the focal 
question (How are scientific and technological developments related to 
climate change?), while including dispersion as part of their answers. An 
understanding of the role of this physical phenomenon in linking local 
actions and global consequences was necessary in order to avoid LIPHs. 
This understanding is particularly important because dispersion and 
transportation of pollutants through the atmosphere are not appropriately 
considered in most discussions about climate changes in mass media 
(Bostrom et al., 1994; Fischhoff, 2007; Lowe et al., 2006). Therefore, the 
propositions established to or from dispersion may indicate the level of 
student understanding about why local actions may result in global 
problems. 
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The data from Table 3 show that almost 70% of the propositions were 
present in Cmaps containing 2 or 3 propositions. One might expect that an 
increase in the number of propositions would indicate a deeper 
understanding of dispersion (the CC), as there are many links with other 
concepts. However, only 2 students used more than 6 propositions, and all 
of them were categorised as limited (L) and inappropriate (I). It is 
probable that these students did not know how to use dispersion to address 
the focal question and instead closely followed the exam instructions, 
making as many propositions to/from the CC as possible. It seems 
plausible to assume that rote learning was responsible for the high number 
of NCs and the low quality of the propositions (Ausubel, 2000; Mayer, 
2002). This possibility demonstrates that instructors can use this method 
as a straightforward indicator to track students who are choosing to use 
rote rather than meaningful learning. This information may be useful in 
planning precise interventions with specific groups of students, in order to 
influence them to choose a more meaningful approach to learning. 

The majority of the propositions (68%) were assigned as VR or R and 
were prevalent in Cmaps containing only 2 or 3 propositions. The students 
who understood the role of dispersion used the CC very selectively to 
respond to the focal question, and connected the CC to other propositions 
only when it was necessary to enrich the network. This approach was 
utilised by skilled map-makers who could express the relationship 
between local actions and global consequences by using dispersion as part 
of their answer. This method is the opposite of rote learning and can also 
be used as an indicator of learning. The existence of few NCs with high 
quality propositions seems to be related to meaningful learning about the 
subject. In summary, a high number of NCs shows a lack of selectivity in 
using the CC appropriately and may be related to rote learning. On the 
other hand, a low number of NCs may indicate accuracy in using the CC 
to set up an appropriate Cmap to address the focal question, which may be 
related to meaningful learning (Cicuto & Correia, 2012). We believe the 
appropriate use of Cmaps in the classroom allows instructors to obtain 
valuable information about student understanding in order to make 
appropriate instructional choices and to foster meaningful rather than rote 
learning. 

The low incidence of L propositions (13%) can be considered to be an 
indicator of student mapping proficiency, as the majority of propositions 
(87%) considered in this study were of high semantic clarity. Therefore, 
the training session applied during classes 1-4 (Figure 3) seems to have 
allowed first-time users to understand how to make good Cmaps. This 
prerequisite is critical in order to ensure a reliable evaluation of the Cmaps 
obtained in the everyday classroom. In our specific case, we can be sure 
that a proposition containing a concept error is due to a misunderstanding 
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about the topic and is not related to the students’ lack of ability to prepare 
a Cmap (Conradty & Bogner, 2010; Aguiar, Cicuto & Correia, 2014). 

Figure 5 shows two illustrative Cmaps to highlight the differences 
between the patterns that emerged after categorising the propositions 
to/from the CC (dispersion). In this first step of the NeAn, the differences 
in quality between the propositions of students’ Cmaps became clear. 
Figure 5a shows a rich Cmap with 3 NCs (shadowed white boxes) and 3 
propositions linking them to the CC (black box). These propositions were 
categorised as very relevant (VR, e.g. dispersion - makes global —> 
environmental problems) or relevant (R, e.g. greenhouse gases - are 
submitted to the process of —> dispersion). On the other hand, Figure 5b 
shows a poor Cmap containing 7 NCs (shadowed white boxes) and 9 
propositions linking them to dispersion. All of them were classified as 
limited (L, e.g. C0 2 - because of the —> dispersion) or inappropriate (I, 
e.g. energy crisis - brings —> dispersion). It is worth noting that the length 
of the linking phrase is another indicator of conceptual understanding. 
Typically, the linking phrases in Figure 5b are longer than in Figure 5a, 
because this student needs to use more words to express his own ideas and 
to connect the concepts. A clearer understanding of the topic allows the 
map-makers to be more concise in describing their thoughts. 

NeAn second step: fast assessment of Cmaps from a classroom 

A quick diagnostic assessment of learning using students’ Cmaps can be 
achieved by using the second step of NeAn. The categorised propositions 
(n=175) were used to calculate a weighted average for each Cmap. The 
goal was to estimate the understanding level of all students at a glance, 
thereby informing the instructor about the presence (or absence) of LIPHs. 
The WAs were used to separate Cmaps into the same categories 
developed in the first step of the NeAn: inappropriate (-2 < weighted 
averages < -1), limited (-1 < weighted averages < 0), relevant (0 < 
weighted averages < 1), and very relevant (1 < weighted averages < 2). 
The results shown in Figure 6 indicate that 60% of the Cmaps could be 
classified as relevant (R) and very relevant (VR), taking into account only 
the propositions using dispersion (CC). These Cmaps (n=41) have no (or 
few) LIPHs involving the CC and the subset VR category (n=25) 
represents the class benchmark. 

Evaluating Cmaps is a complex task because the one-right-answer 
approach is not useful to deal with the multitude of ideas expressed by the 
propositional networks set up by students. Therefore, the identification of 
the best Cmaps in an everyday classroom can help professors to establish 
parameters to more precisely rank student Cmaps. This strategy seems 
more interesting than the use of the instructor’s own Cmap as a model of 
desirable answers. The difference among the zones of proximal 
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development among students (peers) is lower than the difference between 
students and the professor (high asymmetry). Considering the multitude of 
possible propositions stated by the students, it is fairer to compare only 
Cmaps produced by students (Correia & Infante-Malachias, 2009). 
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Figure 5. Students’ Cmaps to illustrate (a) the selective choice of NCs to 
make acceptable propositions, and (b) the pervasive use of NCs 
to make irrelevant propositions using dispersion. Focal question: 
How are scientific and technological developments related to 
climate change? Compulsory concept (CC), neighbour concepts 
(NC) and supplementary concepts (SC) are represented in black, 
white shadowed and white boxes, respectively. The dashed box 
highlights the root concept of the HSCmaps. 
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Categories 

Figure 6. Frequency graph showing the Cmap distribution among the 
categories inappropriate (I), limited (L), relevant (R) and very 
relevant (VR). 

LIPHs were found mainly in the Cmaps (n=18) classified as limited 
(L). A more detailed analysis reveals the mistakes in the concept 
relationships made to/from dispersion (the CC). The precise identification 
of the students’ LIPHs helps the instructor to plan the next activities and 
thereby foster meaningful learning. These students probably need to revise 
some concepts involving climate change; specific recommendations can 
be made by the professor. Only 11% of all Cmaps (n=8) were classified as 
inappropriate (I), which reinforces the importance of the training session 
on concept mapping (Figure 3). 

Figure 7 shows selected Cmaps to highlight the main features of the 
categories used to assess them using the NeAn and to identify LIPHs. The 
Cmap presented in Figure 7a contains only one neighbour concept (NC) 
related to dispersion (CC). Despite being very selective, this student made 
a mistake while expressing the relationship between greenhouse gases 
(NC) and dispersion (CC). The proposition that “greenhouse gases - are 
not submitted to the process of —> dispersion” is the opposite of what is 
considered correct according to current scientific knowledge. This mistake 
reveals the existence of LIPHs, and the proposition is more inappropriate 
than limited. This lack of understanding about the role of dispersion (CC) 
in answering the focal question explains the appearance of this 
inappropriate (I) proposition. 

The Cmap shown in Figure 7b has 2 NCs (C02 and planet) linked to 
dispersion. The propositions elaborated by this student (C02 - reducing 
the —> dispersion, and dispersion - in the —> planet) are not clear enough 
to allow a critical judgement about the messages they express. The 
concepts seem to be in the wrong place ( e.g ., dispersion can reduce the 
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concentration of C0 2 ), and “dispersion in the planet” could be considered 
as a concept, because there is no verb to clarify the relationship between 
these concepts. These propositions reveal the existence of LIPHs, and they 
are more limited than inappropriate (the opposite of the Cmap presented 
in Figure 7a). 

Figures 7c and 7d present Cmaps without LIPHs concerning the 
propositions involving the CC. However, the proposition in Figure 7c 
(greenhouse gases - are submitted to the process of —> dispersion) does 
not address the possible implications of local actions for global 
environmental consequences. This topic, explored during the instructional 
period, is a critical aspect for differentiating relevant and very relevant 
Cmaps. Figure 7d presents a Cmap with 2 NCs and 2 propositions that 
make clear the student’s understanding about the spatial dimension of the 
climate change (dispersion - globalises —> climate change, and dispersion 
- socialises the —> pollution). These statements appear because this 
student understands climate change beyond the naive description 
presented in the mass media. The role of atmosphere dynamics is taken 
into account to explain why we are facing environmental challenges that 
are more complex than those previous ones. Climate change requires a 
global commitment to discovering potential actions to mitigate its 
dangerous effects and to solve this problem during the 21 st century. This 
Cmap reaches the instructor’s expectations and indicates the effectiveness 
of discussions during classes 6-9. It needs to be stressed that 25 Cmaps 
(35%) considered in this study present this feature, indicating the positive 
effect of the instructional activities. 

Conclusions 

Neighbourhood Analysis (NeAn) is a straightforward way to identify 
Limited or Inappropriate Propositional Hierarchies (LIPHs) in Cmaps. 
The choice of the compulsory concept (CC) and the focal question allow 
instructors to find LIPHs through a fast diagnostic assessment of student 
conceptual outcomes. Only a subset of all propositions needs to be 
considered, thereby enabling a rapid evaluation of the Cmaps. Instructors 
can provide specific high-quality feedback to their students, even under 
normal teaching conditions. Cognitive conflicts can be dealt with through 
dialogue among students and the professor, who can address the specific 
misunderstandings of each student. NeAn used in conjunction with 
concept mapping helps the instructor to provide students with scaffolding 
during the course and to influence them to use meaningful, rather than 
rote, learning. In this context, students become aware of the utility of 
concept mapping as a way to learn how to learn, and their empowerment 
can be the most relevant achievement of this process, as suggested by 
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Novak (2002). We also believe that NeAn is a promising strategy to 
teachers tracking the effects of their instructional options. 
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(c) 



(d) 


DISPERSION 



Figure 7. Students’ Cmaps to illustrate the main features of the categories 
used to analyse them: inappropriate (a), limited (b), relevant (c) 
and very relevant (d). Compulsory (CC), neighbor (NC) and 
supplementary (SC) concepts are represented in black, white 
shadowed and white boxes, respectively. The dashed box 
highlights the root concept of the HSCmaps. 
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The results presented here demonstrate the possibility of identify 
patterns in student understanding of the role of dispersion (threshold 
concept) in explaining why local actions may have in global 
consequences. NeAn can be meaningfully applied for assessing science 
students at secondary level and above. The ideas presented in this article 
can be directly applied at the secondary level once students have been 
successfully introduced to the Cmap procedure. NeAn is very applicable 
for biology, chemistry or physics lessons and thus once the Cmap 
technique has been introduced to the students, assessment can take place 
in all science subjects in a similar manner and also, given the potential for 
interdisciplinary assessment, the NeAn idea can have potential 
applicability across science subjects. 
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Appendix 1: List of acronyms 

CC: Compulsory Concept 

Cmaps: Concept Maps 

ECL: Expanded Collaborative Learning 

HSCmap: Half-Structured Concept Map 

I: Irrelevant 

L: Limited 

LIPHs: Limited or Inappropriate Propositional Hierarchies 

NC: Neighbour concept 

NeAn: Neighbourhood Analysis 

PCT: Propositional Clarity Table 

R: Relevant 

SC: Supplementary Concept 
VR: Very Relevant 
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